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q/ Abstract 

Oh ' The behavior of average approximation cardinality for d-parametric random 

r~| ■ fields of tensor product type is investigated. It was shown in [5] that for given rel- 

ative error level the approximation cardinality increases exponentially, as d — > oo, 
i.e. one observes the curse of dimensionality (intractability) phenomenon. 

The aim of this paper is to give the exact asymptotic expression for the approx- 
imation cardinality. 
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^ : 1 Introduction 

Suppose we have a random function X(t), with t in some parametric set T, admitting 
a series representation via random variables and the deterministic real functions (p^, 
namely 

oo 

fe=l 

For any finite set of positive integers K C N let Xx{t) = J2k&K £fc^fc(£)- I n many problems 
one needs to approximate X with finite rank process Xk ■ Natural questions arise then: 
how large should be K that yields a given small approximation error? Given the size 
of K, which K provides the smallest error? In this article, we address the first of these 
questions for a specific class of random functions, namely, tensor product type random 
fields with high- dimensional parameter sets. 

Let (A(z))j>i be a non-negative sequence satisfying 



i) 2 < oo 



i=l 
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and let (y?i)i>o be an orthonormal system in L 2 [0, 1]. 

Consider a family of tensor product type random fields X = £ [0,l] d }, 

d = 1,2, given by 



d d 

fcgjjd i=l Z = l 

where ^ are non-correlated random variables with zero mean and unit variance. Obvi- 
ously, under assumption (II. ip the sample paths of belong to L 2 ([0, almost surely 
and the covariance operator of has the system of eigenvalues 

d 

(1.3) xl^HMk) 2 , keN d . 

i=i 

In the following we drop the index d and write X(t) instead of X^(t). We set T = [0, l] d . 

For any n > 0, let X n be the partial sum of (11.21) corresponding to n maximal eigen- 
values. We study the behavior of average error of approximation to X by X n 

e(X,X n ;d) = (E||X-X n ||i 2(T) ) 1/2 , 

as d — > oo. Since in the following we consider only L 2 (T)-norms, we will simply write 1 1 • 1 1 
instead of || ■ ||x 2 (t)- It is well known (see, for example, [2], [1] or [S]) that X n provides the 
minimal average quadratic error among all linear approximations to X having rank n. 

Since we are going to explore a family of random functions, it is more natural to 
investigate relative errors, that is to compare the error size with the size of the function 
itself. 

Let 

oo 

i=i 

then 

e||x|| 2 = x i = Ad - 

Define the relative average approximation cardinality as 

n(e, d) := min{n : ^[^fi < e} = min{n : E\\X - X n \\ 2 < e 2 A d }. 



{MX 



The study of n(s, d) we are interested in here belongs to the class of problems dealing 
with the dependence of the complexity for linear multivariate problems on the dimension, 
see the works of H. Wozniakovski ([9], [10], [11], [12]) and the references therein. 
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It was suggested in |5j to use an auxiliary probabilistic construction for studying the 
properties of deterministic array of eigenvalues (11.31) . We also follow this approach. 

Consider a sequence of independent identically distributed random variables (U{) , I = 
1, 2, ... with the common distribution given by 

(1.4) P(tf, = -logA(i)) = ^, £ = 1,2, ... 
Under assumption 

oo 

(1.5) ^|logA(*)| 3 A(*) 2 < oo, 

i=l 

the condition E|£7/| 3 < oo is obviously satisfied. 

Let M and a 2 denote the mean and the variance of Ui, respectively. Clearly, 

A(*) 2 



M = -^logA(? 



A 



i=l 
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a 2 = ^|logA(z)| 2 ^ - M 2 . 

i=l 

Then the third central moment of Ui is given by 



a 3 := E(Ui -M) 3 = -J2 ^g\(t) 3 - Mia 2 - M 3 . 

i=i 

If f ll.5p is verified, we have \M\ < oo, < cr 2 < oo and \a\ < oo. 
In the sequel the explosion coefficient 

(1.6) 8 := Ae 2M 

will play a significant role. It was shown in [5] that £ > 1, except for the totally degenerate 
case when the number of strictly positive eigenvalues is zero or one. In other words, S = 1 
iff (7 = 0. Henceforth we will exclude this degenerate case. 
The following result was obtained in [5], Theorem 3.2. 

Theorem 1.1 Assume that the sequence (A(i)) satisfies 

oo 

^|logA«| 2 A(*) 2 <oo. 
i=i 

Then for any e e (0, 1) we have 

lim logw(e,rf) - allogS _ ^ 

where the quantile q — q{e) is chosen from the equation 

(1.7) =e 2 . 
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The authors of [5] conjectured that under further assumptions on the sequence (A(i)) 
one can prove that 

, A C(e)£ d e 2q ^ 
n(e, a) ~ = , a — > oo. 

Indeed, we are going to confirm this conjecture. 

2 Main result 

It turns out that two different cases depending on the nature of the distribution of U\ 
should be distinguished. The proof and the final result rely on the fact whether this 
distribution is a lattice, one or not. 

Recall that one calls a discrete distribution of a random variable U a lattice distri- 
bution, if there exist numbers a and h > such that every possible value of U can be 
represented in the form a + uh, where v is an integer. The number h is called a span of 
the distribution. In the following, when studying the lattice case, we assume that h is a 
maximal span of the distribution, i.e. that it is impossible to represent all possible values 
of Ui in the form b + vh\ for some b and h\ > h. 

Definition (11.41) yields that the variables Ui have a common lattice distribution iff 
= Ce~ Uih for some positive real C, h and rij e N. We call this situation the lattice 
case and will assume that h is chosen to be the largest possible. Otherwise we say that 
the non-lattice case takes place. 

Theorem 2.1 Let the sequence (A(z)), % = 1,2,... satisfy (11. 5p . 
Then for any e G (0, 1) we have 



d)=K 0(i) £ d e 2qV1 d~ 1/2 (1 + o(l)) , d -> oo, 

(7 



where 



(x) = -4e- 2 / 2 , 
ft 



in the lattice case 
otherwise, 



and i/ie quantile q = q(s) is defined in ( 11.71) . 
Remarks: 

• One can see that the complexity of approximation increases exponentially as d — > oo. 
This phenomenon is referred to as "dimensionality curse" or "intractability", see 
e.g. [8] and |10| . The notion of "curse of dimensionality" dates back at least to 
Bellman pp. 
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• Since by l'Hospital rule, 

lim — = - , 

h-o 1 - e~ 2h 2 

the formulae for K agree when h — > 0. 
Proof: 

Let ( = ((e, d) be the maximal positive real number such that the sum of eigenvalues 
satisfies 

keN d -.x k <c 

Define a lattice set in N d 

d 

A = A(e, d) := {k G N d : X k > (} = {k G N d : JJ A(Jfe,) > (}• 

l=i 

Since for any k G A it holds that A& > 0, one can write 

\ 2 



n(e, d) = card (A) = -f 



fceA 

£ A rf exp{-2^1ogA(A; i )}n p (^ = -logAfa)) 

d 

A d Eexp{2^^}I {Etil/j <_ logC} . 



2=1 

For centered and normalized sums 



we have 



where 

(2.8) = 0(e,d) 



Za _ J2tiUi-dM 
ayfd 

d 

{J2u l <-\og(} = {Z d <6}, 
log C + dM 



i=i 



a\fd 

We show now that 9 has a useful probabilistic meaning in terms of (U{) and of their 
sums. Applying Lemma 3.1 from [5] we have for any d G N and zGl 1 

£ A 2 , = A d p(X>>-log^ 

feeN d :A fc <^ \l=l 
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where 

log z + dM 
o\fd 

Fix £ G (0, 1). Observe that 

x l < £2Ad 

fcSN d :A fe <z 

iff 

W(Z d >6 z ) < e 2 . 

Therefore, 9 = 9(s,d) defined by (12. 8p is the (1 — e 2 )-quantile of the distribution of Z d , 
namely, 

9(e, d) = mm{9 : P (Z d > 9) < e 2 } = min{0 : P (Z d < 9) > 1 - e 2 }. 

Let q = q(e) be the quantile of the normal distribution function chosen from the equa- 
tion (jl.7p . Then in view of the Central Limit Theorem 

(2.9) 9(d,e)^^-, d^oo, 

o 

for any fixed e G (0, 1). 

Now let us return to the average approximation cardinality. We obtain 



n(e,d) = S d Eexp{2aVdZ d }I {Zd < e} 

r 6 

= £ d exp{2aVd9} / exp{2aVd(z - 9)} dF d (z), 

J — oo 

where Fd(z) = f(Z d < z) and £ is defined in (11.61) . 
Denote 

^ d {z) := exp{2aVd(z - 9)} 
and integrate by parts the integral 

V d (z) d[F d (z) - F d (9)] = j [-F d (z] + F d {9)\ d* d (z). 



Non-lattice case 

In the following part of the proof we will assume that the distribution of (£//) is not 
a lattice one. This is true in most interesting cases, such as the Brownian sheet, the 
fractional Brownian sheet, the Brownian pillow, the Brownian pillow-slip (Kiefer field). 

In view of (11.51) we are able to apply Cramer-Esseen Theorem (cf. Theorem 2 §42 
in [3], Theorem 5.21 §5.7 of Chapter V in [7j or Theorem 4 §3 of Chapter VI in [6]). It 
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leads to 



/ [-F d (z) + F d (9)]dV d (z) = f [-$(z)+$(9)]d* d (z) 

J— oo J-oo 



= Ii + 1 2 - ^3 - h + o 

where 

h = I [-$(*) + $(0)]d* rf (*), 

J —oo 



1 



OO 

J 4 = Q ° (fl 2 - l) e^ 2 ~ Q ° ( ffl 2 - 1 ) e~^ 2 , d 



oo. 



The last equivalence is provided by (12. 9p . 

Since d$ d (z) = 2a\fdty d (z)dz, then I2 is given, after a change of variable, by 

where y = —yd(z — 9). 
For any d — 1, 2, ... 



2 



This estimate gives us the majorant required in Lebesgue Theorem. Using (I2.9p and 
passing to the limit in the integral we obtain, as d — > 00, 

Similarly, 

Thus we obtain that = Vd(h — h) (1 + °(1))> hence, I 2 — I3 — I4 = o (^n ■ 

Consider the main integral I\. 
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h = h(d,6) = I [-$(*) +$(0)]d* d (z) 

J — oo 

1 f e 

= —= / exp{2cx\/d(z - 9)}exp{-z 2 /2} dz 

V27T J-oo 



oo 
oo 



1 y 

ex P{~n( 7^) 2 } ex P{- 2o "y} dy 



\f2~nd Jo 2 ^fd 

1 

2(Tv / 27rrf 



f2.ll) ~ 1 e - q 2 /2*2 d ^OO. 



n M ) = ^y^ expF^Hl + °(D) 



Then 



as asserted. 

Lattice case 

Now we will proceed under assumption that the sequence ([//) has a lattice distribution. 
Let the possible values of the random variable Ui be 

a + uh, v = 0, ±1, ±2, ... 

where a = M + a is the shift, and h is the maximal span of the distribution. Therefore, 
all possible values of Z d have the form 

da + uh 
ay/d 

Introduce the function 



u = 0,±1,±2,... 



S(x) = [x] - x + -, 
where [x] denotes, as usual, the integer part of x, and consider 

, . h f xo\fd — da 
SAx) = - S 



<7 \ h 



Let Fd(z) be as above. Then under assumption (11.51) Esseen's result (see Theorem 1 § 43 
in [3]) yields 



e~ z2 l 2 fSAz) a 3 (z 2 -l)\ ( 1 



uniformly in z. 

Comparing with (I2.10p , we observe that one only needs to evaluate the additional term 
J = -J= J 9 [-S d {z)e- z2 ' 2 + S d (0)e- e2 / 2 )d* d (z) 



-4=, f ^d{z)d (S d {z)e- z2 ' 2 ) = J, - J 2 + J 3 , 



where 



Ji = ^mS-J d{z)s ' d{z)e ~ z212 ^ 

J 2 = -^=J y d (z)S d (z)ze- z2 / 2 dz, 



and J 3 is a " discrete part" , which is defined in the following way. 

Notice that S(x) is a periodic function with period one, therefore Sd(x) possesses a 
period h/aVd and has jumps at points { kh ^j^ , k G Z}. If the point 9 belongs to this 
lattice then there exists an integer k' such that 9 = k '^^ a ■ Hence, one can integrate the 
discontinuous part of the integral J with respect to the measure -S kh+da and obtain 

<y\fd 

1 h sr^ t ( kh + da\ , 1 fkh + da\ , 

We start with the estimation of J±. At the points where the derivative S' d (z) makes 
sense, one can easy calculate that S' d (z) = ^ za ^~ da ^j — — y^d, therefore, similarly to 
the non-lattice case, by Lebesgue Theorem we have 



j -Vd 
1 V2nd 



/ exp{2aVd(z - 9)} exp{-^ 2 / 2 }d^ 

J — OO 



-oo 

r-OO 



V2nd Jo 

[2.12) ~ ~ l ^ e- q2 ' 2 ° 2 , d -> oo, 



and it yields v^^i = —\fd~h (1 + o(l)). 

As for the integral J 2 , this one, as d is large enough, plays no role at all. Indeed, 

i r e 

J 2 = —;= \ exp{2aVd(z - 9)}S d (z)zexp{-z 2 /2}dz 



3h 

2<rdy/2n 
3h 



oo.. 



And, of course, J2 — ■ 
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Now we consider the most essential summand 

k' 



J, 



(2.13) 
We obtain 



1 h r n ( kh + da A, r 



1 f kh + da 



k=—oc 
k 



ay/d 



1 h 

1 /l 



fc=— oo 

OO 



exp{— 2/iZ} exp{- 



1 f(k' - l)h + da 



1=0 
oo 



1 



oo / 

^exp{-2M}exp{--^ 



Ih 



1=0 

h 



a 



Vd 



a^fd (1 - e- 



2M 



' ^ , d 



2tt 



oo. 



vdJs = \/d- 



2/i 



h (l + o(l)). 



(1 -e- 2fe 7 

Putting together (EOT]) . fl2~T2D and (l2~T3l) . we obtain 



n(e, d) 



1 



v/d (1-e- 



e" 9 / 2CT (1+o(1)), d^oo. 
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